Image enhancement method and apparatus, and storage medium

ABSTRACT

Embodiments of this disclosure include an image enhancement method and apparatus. The image enhancement may include obtaining an original image and performing synthesis processing on features of the original image to obtain a first illumination map corresponding to the original image. A resolution of the first illumination map may be lower than a resolution of the original image. The image enhancement may further include obtaining, based on the first illumination map, a mapping relationship between an image to an illumination map and performing mapping processing on the original image based on the mapping relationship to obtain a second illumination map. A resolution of the second illumination map may be equal to the resolution of the original image. The image enhancement may further include performing image enhancement processing on the original image according to the second illumination map to obtain a target image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of PCT Patent ApplicationNo. PCT/CN2020/075472, entitled “IMAGE ENHANCEMENT METHOD AND APPARATUS,AND STORAGE MEDIUM” filed on Feb. 17, 2020, which claims priority toChinese Patent Application No. 201910148574.6, entitled “IMAGEENHANCEMENT METHOD AND APPARATUS, AND STORAGE MEDIUM” filed with theNational Intellectual Property Administration, PRC on Feb. 28, 2019,wherein the content of each of the above-referenced applications isincorporated herein by reference in its entirety.

FIELD OF THE TECHNOLOGY

This application relates to the field of image processing, andspecifically, to an image enhancement method and apparatus, and astorage medium.

BACKGROUND OF THE APPLICATION

In recent years, with the improvement of photographing technologies ofelectronic devices, requirements on the image quality have alsoincreased. Underexposure caused by insufficient light or backlightaffects the quality of an image, frustrates an effort to capture desireddetails, and may lead to other problems. Therefore, an image enhancementmethod may be used to improve the image quality. In the current imageenhancement method, a network model is trained using pairs of originalimages and annotated images, to obtain a network model that can enhancean image, but this method has low efficiency in training the networkmodel.

SUMMARY

In view of this, embodiments of this disclosure provide an imageenhancement method and apparatus, and a storage medium, which canimprove the efficiency of image enhancement.

In a first aspect, an embodiment of this disclosure provides an imageenhancement method. The method may include obtaining an original imageand performing synthesis processing on features of the original image toobtain a first illumination map corresponding to the original image. Aresolution of the first illumination map may be lower than a resolutionof the original image. The method may further include obtaining, basedon the first illumination map, a mapping relationship between an imageand an illumination map and performing mapping processing on theoriginal image based on the mapping relationship to obtain a secondillumination map. A resolution of the second illumination map may beequal to the resolution of the original image. The method may furtherinclude performing image enhancement processing on the original imageaccording to the second illumination map to obtain a target image.

In a second aspect, an embodiment of this disclosure provides an imageenhancement apparatus. The image enhancement apparatus may include amemory operable to store computer-readable instructions and a processoroperable to read the computer-readable instructions. The processor whenexecuting the computer-readable instructions may be configured to obtainan original image and perform synthesis processing on features of theoriginal image to obtain a first illumination map corresponding to theoriginal image. A resolution of the first illumination map may be lowerthan a resolution of the original image. The processor may further beconfigured to obtain, based on the first illumination map, a mappingrelationship between an image and an illumination map and performmapping processing on the original image based on the mappingrelationship to obtain a second illumination map. A resolution of thesecond illumination map may be equal to the resolution of the originalimage. The processor may be further configured to perform imageenhancement processing on the original image according to the secondillumination map to obtain a target image.

In a third aspect, an embodiment of this disclosure provides anon-transitory computer-readable storage medium storing processorexecutable instructions. The instructions may cause a processor toobtain an original image and perform synthesis processing on features ofthe original image to obtain a first illumination map corresponding tothe original image. A resolution of the first illumination map may belower than a resolution of the original image. The processor may furtherbe configured to obtain, based on the first illumination map, a mappingrelationship between an image and an illumination map and performmapping processing on the original image based on the mappingrelationship to obtain a second illumination map. A resolution of thesecond illumination map may be equal to the resolution of the originalimage. The processor may be further configured to perform imageenhancement processing on the original image according to the secondillumination map to obtain a target image.

In the embodiments of this disclosure, an original image is obtained;synthesis processing is performed on features of the original image toobtain a first illumination map corresponding to the original image, aresolution of the first illumination map being lower than a resolutionof the original image; a mapping relationship for mapping an image to asecond illumination map is obtained based on the first illumination map;mapping processing is performed on the original image based on themapping relationship to obtain a second illumination map, a resolutionof the second illumination map being equal to the resolution of theoriginal image; and image enhancement processing is performed on theoriginal image according to the second illumination map to obtain atarget image. This solution can improve the efficiency of imageenhancement.

BRIEF DESCRIPTION OF THE DRAWINGS

To describe the technical solutions in the embodiments of thisdisclosure more clearly, the following briefly describes theaccompanying drawings required for describing the embodiments.Apparently, the accompanying drawings in the following description showmerely some embodiments of this disclosure, and a person in the art maystill derive other drawings from these accompanying drawings withoutcreative efforts.

FIG. 1 is a schematic diagram of an application scenario of an imageenhancement method according to an embodiment of this disclosure.

FIG. 2 is a first schematic flowchart of an image enhancement methodaccording to an embodiment of this disclosure.

FIG. 3 is a second schematic flowchart of an image enhancement methodaccording to an embodiment of this disclosure.

FIG. 4 is a third schematic flowchart of an image enhancement methodaccording to an embodiment of this disclosure.

FIG. 5 is a second illumination map according to an embodiment of thisdisclosure.

FIG. 6 is a schematic structural diagram of a convolutional networkaccording to an embodiment of this disclosure.

FIG. 7 is a schematic flowchart of image enhancement according to anembodiment of this disclosure.

FIG. 8 is a schematic flowchart of input and output of an imageenhancement method according to an embodiment of this disclosure.

FIG. 9 is a schematic diagram of a first experimental result accordingto an embodiment of this disclosure.

FIG. 10 is a schematic diagram of a second experimental result accordingto an embodiment of this disclosure.

FIG. 11 is a schematic diagram of a third experimental result accordingto an embodiment of this disclosure.

FIG. 12 is a schematic diagram of a fourth experimental result accordingto an embodiment of this disclosure.

FIG. 13 is a schematic diagram of a fifth experimental result accordingto an embodiment of this disclosure.

FIG. 14 is a schematic diagram of a sixth experimental result accordingto an embodiment of this disclosure.

FIG. 15 is a first schematic structural diagram of an image enhancementapparatus according to an embodiment of this disclosure.

FIG. 16 is a second schematic structural diagram of an image enhancementapparatus according to an embodiment of this disclosure.

FIG. 17 is a schematic diagram of a network device according to anembodiment of this disclosure.

DESCRIPTION OF EMBODIMENTS

Referring to the drawings, same component symbols represent samecomponents. A principle of this disclosure is described using examplesin which this disclosure is implemented in proper computingenvironments. The following descriptions are specific embodiments ofthis disclosure based on the examples, and are not to be construed as alimitation to other specific embodiments of this disclosure that are notdescribed herein in detail.

In the following description, the specific embodiments of thisdisclosure are described with reference to steps and symbols ofoperations that are performed by one or more computers, unless indicatedotherwise. Therefore, such steps and operations, which are at timesreferred to as being computer-executed, include the manipulation by theprocessing unit of the computer of electrical signals representing datain a structured form. This manipulation transforms the data or maintainsthe data at locations in the memory system of the computer, whichreconfigures or otherwise alters the operation of the computer in amanner well understood by a person skilled in the art. Data structuresin which the data is maintained are physical locations of the memorythat have particular properties defined by the format of the data.However, while the principle of this disclosure is being described inthe foregoing text, it is not meant to be limiting as a person skilledin the art will appreciate that the various steps and operationsdescribed hereinafter may be implemented in hardware.

The term “module” (and other similar terms such as unit, submodule,etc.) may refer to a software module, a hardware module, or acombination thereof. A software module (e.g., computer program) may bedeveloped using a computer programming language. A hardware module maybe implemented using processing circuitry and/or memory. Each module canbe implemented using one or more processors (or processors and memory).Likewise, a processor (or processors and memory) can be used toimplement one or more modules. Moreover, each module can be part of anoverall module that includes the functionalities of the module.

In this disclosure, the terms “first”, “second”, “third”, and the likeare intended to distinguish between different objects but do notindicate a particular order. In addition, the terms “include”, “have”,and any variant thereof are intended to cover a non-exclusive inclusion.For example, a process, method, system, product, or device that includesa series of steps or modules is not limited to the listed steps ormodules. Instead, some embodiments further include a step or module thatis not listed, or some embodiments further include another step ormodule that is intrinsic to the process, method, product, or device.

“Embodiment” mentioned in the specification means that particularfeatures, structures, or characteristics described with reference to theembodiment may be included in at least one embodiment of thisdisclosure. The term appearing at different positions of thespecification may not refer to the same embodiment or an independent oralternative embodiment that is mutually exclusive with anotherembodiment. A person skilled in the art explicitly or implicitlyunderstands that the embodiments described in the specification may becombined with other embodiments.

An embodiment of this disclosure provides an image enhancement method.The image enhancement method may be performed by an image enhancementapparatus provided in an embodiment of this disclosure, or a networkdevice integrated with the image enhancement apparatus. The imageenhancement apparatus may be implemented in manner of hardware orsoftware. The network device may be a device such as a smartphone, atablet computer, a palmtop computer, a notebook computer, or a desktopcomputer.

FIG. 1 is a schematic diagram of an application scenario of an imageenhancement method according to an embodiment of this disclosure. FIG. 1takes the image enhancement apparatus being integrated in the networkdevice 100 as an example. The network device 100 can obtain an originalimage 101; perform synthesis processing on features 102 of the originalimage 101 to obtain a first illumination map 103 corresponding to theoriginal image 101, a resolution of the first illumination map 103 beinglower than a resolution of the original image 101; obtain, based on thefirst illumination map 103, a mapping relationship for mapping an imageto a second illumination map 104; perform mapping processing on theoriginal image 101 based on the mapping relationship to obtain a secondillumination map 104, a resolution of the second illumination map 104being equal to the resolution of the original image 101; and performimage enhancement processing on the original image 101 according to thesecond illumination map 104 to obtain a target image 105.

FIG. 2 is a schematic flowchart of an image enhancement method accordingto an embodiment of this disclosure. Referring to FIG. 2, the imageenhancement method according to an embodiment of this disclosure is asfollows.

201: Obtain an original image.

The original image is an image that needs image enhancement. Originalimages may be images obtained in a variety of image capturingsituations. For example, the original image may be a normally exposedimage, an underexposed image, an image with insufficient light, or abacklit image during image capturing. Content included in the originalimage is not limited.

Due to the diversification of image capturing situations of the originalimages, the image enhancement method can perform image enhancement onthe original images in a variety of image capturing situations, and isnot limited to image enhancement for the normally exposed image, therebyexpanding the application scope of the image enhancement method.

There are many ways to obtain the original image. For example, theoriginal image may be obtained from local storage, a network sidedevice, or the like.

In an embodiment, for example, when imaged are captured by a cameradevice, a currently captured image may be selected as the originalimage. In another example, when images are captured by a camera deviceand displayed in an image capture interface (such as an image previewinterface), an image currently displayed on the interface may be croppedas the original image.

In an embodiment, the original image may alternatively be obtained froma local or external storage unit. For example, the original image mayalternatively be obtained from a local image database.

The image enhancement can enhance useful information in an image tocorrespondingly improve the visual effect of the image for anapplication scenario of the image. The image enhancement can sharpen anoriginally unclear image by purposefully emphasizing a global feature orlocal feature of the image. Alternatively, the image enhancement canenlarge a difference between features of different objects in an imageby emphasizing features of interest. Alternatively, the imageenhancement can meet requirements of special analyses by suppressingfeatures not of interest, improving image quality, enriching imageinformation, and enhancing image interpretation and recognition effects.

202: Perform synthesis processing on features of the original image toobtain a first illumination map corresponding to the original image.

As shown in FIG. 5, an illumination map is a shadow map afterdecomposition of an intrinsic image. An intrinsic image includes areflectance image and a shading image obtained by decomposing anoriginal image. The shading image is an image that reflects lightingconditions of the original image, and the reflectance image refers to animage part that remains unchanged under changing lighting conditions,that is, an image obtained after highlight is removed from the originalimage.

A resolution of the first illumination map is lower than a resolution ofthe original image. The first illumination map may be an image form ofan illumination map with one resolution. The first illumination map maybe an illumination map with a resolution lower than a resolution of theoriginal image. For example, the first illumination map may be alow-resolution illumination map.

The image resolution represents the amount of information stored in animage, and may be expressed by the quantity of pixels per inch in theimage.

Currently, when the deep learning method is used to enhance an image, anetwork model obtained by regression learning from an original image toan annotated image is usually used to perform an image enhancementoperation. However, this method leads to low learning efficiency andpoor robustness of the network model, and defects in image contrast.

In an embodiment, an image enhancement network model obtained byregression learning from an original image to an illumination map may beused to perform the image enhancement operation. The image enhancementnetwork model obtained by regression learning from the original image tothe illumination map has high learning efficiency and strong robustness,and facilitates further operations on the image.

The image enhancement method is applicable to the image enhancementnetwork model. The image enhancement network model uses a mappingrelationship between the original image and the illumination map toreplace a mapping relationship between the original image and theannotated image. The advantage of this approach is that the mappingbetween the original image and the illumination map usually has arelatively simple form and is known a priori. The image enhancementnetwork model thus has a strong generalization ability, and caneffectively process original images obtained in different situationsunder complex photography conditions.

In practical applications, the first illumination map corresponding tothe original image may be obtained through feature synthesis. Forexample, features of the original image may be extracted first, andfeature synthesis is performed on the extracted features to generate thefirst illumination map.

In an embodiment, for example, features of the original image may beextracted using a network structure including a convolution operation.

In the traditional method, the method of adjusting a distribution curveof an image histogram is usually used to enhance the image globally.However, this method causes problems such as locally over-brightness,over-exposure, and over-darkness, and the color of the generated imagemay not be very bright.

In addition, to enhance an underexposed image, it is necessary to adjustlocal features (such as contrast, detail definition, shadow andhighlight) and global features (such as color distribution, averagebrightness, scenario category) of the image at the same time. Therefore,the accuracy of image enhancement can be improved by separatelyextracting the local features and global features of the original image.

In an embodiment, specifically, the step of performing synthesisprocessing on features of the original image to obtain a firstillumination map corresponding to the original image may include:

extracting a local feature and a global feature of the original imagebased on a convolutional network; and

performing feature synthesis on the local feature and the global featureto obtain the first illumination map corresponding to the originalimage.

The convolutional network is a network structure that can extractfeatures of an image. For example, a convolutional network may include aconvolutional layer, and the convolutional layer can extract features ofan image through a convolution operation.

The local feature of the image is a local expression of an imagefeature, and the local feature of the image can reflect a localcharacteristic of the image. For example, local features of an image mayinclude contrast, detail definition, shadow, highlight, and the like.

The global feature of the image can represent an overall feature of theimage. The global feature is relative to the local feature and may beused to describe an overall feature such as a color or shape of an imageor a target. For example, global features of an image may include colordistribution, average brightness, scenario category, and the like.

In practical applications, for example, an original image may beinputted into a convolutional network to extract a local feature and aglobal feature of the original image, and then feature synthesis isperformed on the extracted local feature and global feature to obtain afirst illumination map.

To improve the accuracy of image feature extraction, a network model maybe used to extract image features.

In an embodiment, specifically, the step of extracting a local featureand a global feature of the original image based on a convolutionalnetwork may include:

inputting the original image to the convolutional network;

performing a convolution operation on the original image based on theprimary feature extraction network to extract a primary feature of theoriginal image;

performing a convolution operation on the primary feature based on thelocal feature extraction network to extract the local feature; and

performing a convolution operation on the primary feature based on theglobal feature extraction network to extract the global feature.

As shown in FIG. 6, the convolutional network 600 may include a primaryfeature extraction network 601, a local feature extraction network 602,and a global feature extraction network 603, the local featureextraction network 602 being connected in parallel with the globalfeature extraction network 603, and being connected in series with theprimary feature extraction network 601.

The primary feature extraction network 601 is a network model that canextract a primary feature of an original image. For example, the primaryfeature extraction network 601 may include a pre-trained VGG16 networkmodel. The VGG16 network model may include a 16-layer structure. Forexample, the VGG16 network model may include a convolutional layer, afully connected layer, a pooling layer, and the like.

In an embodiment, a convolution operation may be performed on theoriginal image based on the pre-trained VGG16 network model to extractthe primary feature of the original image. For example, the originalimage can be inputted into the VGG16 network model, and the convolutionoperation can be performed through the convolution layer. Each time theimage is scanned through a convolution kernel, a new matrix isgenerated. After that, the size of a parameter matrix is reduced throughthe pooling layer, thereby reducing the quantity of parameters in thefinal fully connected layer. Then the primary feature of the originalimage is extracted through the fully connected layer.

In an embodiment, according to actual conditions, other types of networkmodels including several convolutional layers may be selected to performa convolution operation on the original image to extract the primaryfeature of the original image.

The local feature extraction network 602 is a network model that canextract a local feature of an image. For example, the local featureextraction network 602 may include two convolutional layers, and theconvolutional layer may be used to extract a local feature. The globalfeature extraction network 603 is a network model that can extract aglobal feature. For example, the global feature extraction network 603may include two convolutional layers and three fully connected layers,and the convolutional layer may be used to extract a global feature.

In practical applications, for example, an original image can beinputted into the primary feature extraction network 601 to extract aprimary feature of the original image. Then, the primary feature isinputted at the same time into the local feature extraction network 602and the global feature extraction network 603 that are connected inparallel, to extract a local feature and a global feature.

To process a high-resolution image in real time, most networkcalculations may be performed in a low resolution condition. Forexample, a resolution of an image may be converted by downsampling.

In an embodiment, specifically, the step of performing synthesisprocessing on features of the original image to obtain a firstillumination map corresponding to the original image may include:

downsampling pixels of the original image to obtain an input image; and

performing synthesis processing on features of the input image to obtainthe first illumination map corresponding to the original image.

The image may be scaled down by downsampling so that the image fits thesize of a display region, and a thumbnail of the corresponding image isgenerated. For example, for an image with a size of m*n, alow-resolution image with a size of (m/s)*(n/s) can be obtained byperforming downsampling with a sampling rate of s on the image, where sis a common divisor of m and n. When considering image pixels in amatrix form, the downsampling of the image is to turn image pixels in ans*s window into one image pixel. The value of the pixel may be anaverage value of all image pixels in the s*s window, or the value of thepixel may be obtained by other calculation methods according to actualsituations.

In practical applications, for example, a matrix with a preset size ofs*s may be obtained from a matrix composed of pixels of the originalimage, and then the pixels in the matrix with the preset size may beconverted into one pixel. The pixel may be obtained according to apreset rule. For example, the pixel may be an average value of allpixels in the matrix with the preset size. After the pixels of theentire original image are converted, an input image whose resolutionafter downsampling is lower than the resolution of the original imagecan be obtained. After that, the input image may be inputted into theconvolutional network for feature extraction, and subsequent steps maybe performed.

203: Obtain, based on the first illumination map, a mapping relationshipfor mapping an image to a second illumination map.

A resolution of the second illumination map is equal to the resolutionof the original image. The second illumination map may be an image formof an illumination map with one resolution. As shown in FIG. 5, thesecond illumination map may be an illumination map with the sameresolution as the original image. For example, the second illuminationmap may be referred to as an original-resolution illumination map.

The mapping relationship may map an image to an illumination map. Forexample, an original image may be mapped to the second illumination mapthrough the mapping relationship. For example, the mapping relationshipmay be a matrix mapping relationship, that is, a mapping transformationmatrix. Mapping transformation between images may be implemented throughsuch a mapping relationship. For example, mapping transformation may beperformed on an original image according to the mapping transformationmatrix to obtain a second illumination map.

In practical applications, the mapping relationship may be obtainedbased on the first illumination map.

To improve the accuracy of image enhancement, the mapping relationshipmay be obtained based on the first illumination map in a bilateral gridmanner.

In an embodiment, specifically, the step of obtaining a mappingrelationship based on the first illumination map may include:

sampling pixels of the first illumination map to obtain sampled pixels;and

mapping the sampled pixels to a bilateral grid to obtain the mappingrelationship.

The bilateral grid is a way of sampling a spatial domain and abrightness domain of the image and mapping the image to the grid. Theterm “bilateral” in the bilateral grid means space and brightness. Afterdiscrete processing, coordinates and brightness information of eachpoint in the image are rounded to the corresponding grid. Throughfiltering and other processing in the grid, combined with interpolationby the method of upsampling, a processed image can be obtained.

For example, spatial-domain and range-domain sampling may be performedon the pixels of the first illumination map, to obtain the sampledpixels. Then positions of the corresponding pixels in the grid arefound, and a grid difference operation is performed to obtain themapping transformation matrix.

In an embodiment, specifically, the step of obtaining, based on thefirst illumination map, a mapping relationship for mapping an image to asecond illumination map may include: obtaining, using a preset trainingimage and a sample enhanced image corresponding to the training image,the mapping relationship that enables loss information between apredicted enhanced image corresponding to the training image and thesample enhanced image to meet a preset condition, the predicted enhancedimage being an enhanced image obtained by performing mapping processingon the training image using the mapping relationship.

In some embodiments, the loss information may be at least one of acontrast loss information, a smoothness loss information, or a colorloss information.

The contrast loss information may be obtained by calculating anEuclidean distance between the predicted enhanced image and the sampleenhanced image, or may be obtained by calculating a Euclidean distancebetween the training image and a restored image. The restored imagerefers to an image obtained by performing inverse mapping (that is,removing the enhancement effect) on the sample enhanced image using themapping relationship.

The smoothness loss information may be obtained by summing spatialvariations (for example, variations in directions of the space) ofvalues of three color channels at each pixel of the mappingrelationship.

The color loss information may be obtained by summing similaritiesbetween color vectors of each pixel in the predicted enhanced image andin the sample enhanced image. The color vector refers to a vectorcomposed of color components (for example, R, G, and B components) ofeach pixel.

By simultaneously optimizing each color channel of the image, the effectof image enhancement can be improved.

204: Perform mapping processing on the original image based on themapping relationship to obtain a second illumination map.

To improve the accuracy of image enhancement, mapping processing may beperformed on the original image in a bilateral grid manner.

In an embodiment, specifically, the step of performing mappingprocessing on the original image based on the mapping relationship toobtain a second illumination map may include:

performing mapping processing on the original image based on the mappingrelationship to obtain a mapped image; and

upsampling the mapped image to obtain the second illumination map.

The principle of bilateral grid upsampling is to select a referenceimage, perform spatial-domain and range-domain sampling on pixels in anyspace of the reference image, then find positions of the pixels in thegrid, and calculate a brightness of an unknown range using a method oftrilinear interpolation.

In practical applications, for example, as shown in FIG. 7, a mappingtransformation matrix may be obtained according to the firstillumination map, and then mapping processing may be performed on theoriginal image 701 using the mapping transformation matrix to obtain amapped image, which is an image with a lower resolution than theoriginal image 701. After that, bilateral grid upsampling 706 may beperformed on the mapped image, and based on pixels of the mapped image,a suitable interpolation algorithm is used to interpolate new elementsbetween the pixels to obtain the second illumination map.

205: Perform image enhancement processing on the original imageaccording to the second illumination map to obtain a target image.

The target image may be an image obtained after the image enhancement.

The problem of image enhancement can be regarded as a problem of findinga mapping relationship between the original image and the target image.For example,

may be used to represent a matrix corresponding to the target image, Imay be used to represent a matrix corresponding to the original image,and a function F may be used to represent a mapping function between theoriginal image and the target image. Then the mapping function F may beexpressed by the following formula:

=F(I)

The target image, the original image, and the second illumination mapare related to each other. For example, S may be used to represent amatrix corresponding to the second illumination map,

may be used to represent the matrix corresponding to the target image,and I may be used to represent the matrix corresponding to the originalimage. Then the relationship among the target image, the original image,and the second illumination map can be shown as follows:

I=S*

Therefore, the target image can be obtained according to the originalimage and the second illumination map. According to the original image Iand the second illumination map S, the obtained target image

can be shown in the following formula:

F(I)=S ⁻¹ ·I

In an embodiment, for example, as shown in FIG. 7, the original image701 may be obtained first, and the original image 701 may be downsampledto obtain an input image of 256×256 pixels. Then the input image isinputted into the primary feature extraction network 702 including thepre-trained VGG16 network model to extract the primary feature of theoriginal image 701, then the primary feature is inputted separately intothe local feature extraction network 703 and the global featureextraction network 704 that are connected in parallel, and the localfeature and the global feature are extracted and merged to obtain thefirst illumination map. Then, the mapping relationship is obtainedthrough bilateral grid upsampling 706, and the second illumination mapis obtained according to the mapping relationship. Finally, the targetimage 709 is obtained through the formula I=S*

. The process of image enhancement 708 can be accelerated by this imageenhancement method, thereby improving the efficiency of imageenhancement.

In an embodiment, the image enhancement method may further include atraining process of an image enhancement network model. As shown in FIG.3, the image enhancement method may further include the followingprocedure.

301: Obtain, based on an image enhancement network model and a trainingimage, a predicted enhanced image corresponding to the training image.

The training image may be an image used by the network model in thetraining process, and the training image includes a sample enhancedimage. The sample enhanced image is an annotation related to imageenhancement performed on the training image.

There are many ways to obtain the training image. For example, thetraining image can be obtained from local storage or a network sidedevice, or may be taken by an image capture device.

There are many ways to annotate the training image. For example, thetraining image may be annotated by an expert to obtain the sampleenhanced image.

The predicted enhanced image may be an image obtained after the trainingimage is enhanced by a network model. There may be a difference betweenthe predicted enhanced image and the actual sample enhanced image, butthe difference can be reduced by training the network model.

In practical applications, for example, the training image may beinputted into the image enhancement network model to obtain thepredicted enhanced image corresponding to the training image. The imageenhancement method for enhancing a training image through the imageenhancement network model is equal to the image enhancement method forenhancing an original image through the image enhancement network model,which has been described above and is not repeated here.

In an embodiment, it is also possible to increase the diversity oftraining samples by randomly cropping a training image. For example,alternatively, the training image may be randomly cropped into aplurality of images of 512×512 pixels to increase the diversity ofsamples.

The accuracy of the network model can be improved by increasing thediversity of the training image. The training image may include imagesin various image capturing situations, such as an image of normalexposure, underexposure, insufficient light, or backlight. The networkmodel trained based on such training images can adapt to images obtainedin different capturing situations in reality.

In an embodiment, for example, by obtaining a standard condition datasetand a special condition dataset, training images including a pluralityof image capturing types may be constructed according to the standardcondition dataset and the special condition dataset.

The standard condition dataset is a dataset that includes normallyexposed images. For example, the standard condition dataset may be aMIT-Adobe Five K Dataset, which includes a plurality of images in rawformat taken by a group of different photographers with a single-lensreflex (SLR) camera, which means that all information recorded by thecamera sensor will be saved. The images cover a wide range of scenes,subjects and lighting conditions. Afterwards, the captured images areretouched with dedicated image adjustment software to obtain thestandard condition dataset.

In an embodiment, for example, the standard condition dataset may be theMIT-Adobe Five K Dataset, and annotations of Expert C may be selected asannotations of the training samples in the dataset. However, because thestandard condition dataset is created mainly to enhance general imagesrather than underexposed images, the standard condition dataset onlyincludes a small portion (about 4%) of unexposed images. As a result,the standard condition dataset lacks images taken in special imagecapturing conditions, such as images captured at night or imagesobtained under non-uniform lighting conditions. To increase thediversity of samples, a special condition dataset may be introduced.

The special condition dataset is a dataset that includes abnormallyexposed images. For example, the special condition dataset may includeimages captured in special image capturing conditions, such asunderexposure, insufficient light, or backlight. Such a specialcondition dataset may include various image capturing situations,scenes, themes, and styles. The added special condition dataset cansupplement image types that the standard condition dataset lacks.

For example, a camera may be used to capture an image with a resolutionof 6000×4000, and then about 15% of images may be collected from animage sharing database using “underexposure”, “insufficient light”,“backlight” and the like as keywords for searching. After that, theexpert use graphics tool software to retouch each collected image toobtain corresponding reference images and establish the specialcondition dataset. Finally, the images in the dataset may be randomlydivided into two subsets, where 2750 images are used for network modeltraining, and 2750 images are used for network model testing.

Training the network model based on the training images constructed bythe standard condition dataset and the special condition dataset canmake the trained network model adapt to various image capturingsituations, thereby improving the accuracy of image enhancement.

302: Obtain loss information between the predicted enhanced image and asample enhanced image based on a target loss function.

The loss information may include one or more of contrast lossinformation, color loss information, and smoothness loss information.The loss information may represent a difference between the predictedenhanced image and the sample enhanced image. The difference may bereduced by training the network model.

The loss function can be used to estimate the degree of inconsistencybetween a predicted value and a true value of the network model. Asmaller value of the loss function indicates better robustness of thenetwork model.

In practical applications, for example, the loss information between thepredicted enhanced image and the sample enhanced image may be obtainedthrough the target loss function. The loss information may be adifference between the predicted enhanced image and the sample enhancedimage, and may be reduced by training the network model.

The target loss function may be flexibly set according to actualapplication requirements. At present, an image is usually enhanced byadjusting an illumination map of the image and performing localsmoothing optimization operations on the illumination map. However, sucha method may leave traces of manual changes of the halo and cause localoverexposure of the image, resulting in excessive image enhancement.

In an embodiment, therefore, a target loss function may be designed, andthe target loss function may include one or more of a reconstructionloss function, a local smoothing loss function, and a color lossfunction. By constraining the illumination map, the image is notover-exposed or over-enhanced.

Specifically, the image enhancement method may further include:

obtaining, based on the image enhancement network model and a trainingimage, a predicted enhanced image corresponding to the training image;

obtaining contrast loss information between the predicted enhanced imageand a sample enhanced image based on a reconstruction loss function, thesample enhanced image being an enhanced image corresponding to thetraining image; and

converging the predicted enhanced image and the sample enhanced imagebased on the contrast loss information to obtain a trained imageenhancement network model.

The reconstruction loss function may be used to obtain the contrast lossinformation of the image. For example, the reconstruction loss functionmay be obtained by measuring a Euclidean distance error. That is, aEuclidean distance between the predicted enhanced image generated by theimage enhancement network model and the sample enhanced image annotatedby the expert is calculated.

The Euclidean distance is a straight-line distance between two points inEuclidean space.

In an embodiment, for example, the reconstruction loss function may beobtained according to a Euclidean distance error metric. For example, Smay be used to represent an original-resolution illumination map matrixcorresponding to the predicted enhanced image,

may be used to represent the sample enhanced image, and I_(i) may beused to represent the training image. The original-resolutionillumination map matrix S corresponding to the predicted enhanced imagemay be multiplied by the sample enhanced image

to calculate the Euclidean distance error metric with the training imageI_(i) to obtain the reconstruction loss function. A formula of thereconstruction loss function L^(i) _(r) may be as follows:

L ^(i) _(r) =∥I _(i) −S×

∥ ²

A multi-channel illumination range may be (I_(i))_(c)≤(S)_(c)≤1, allpixel channels in the sample enhanced image

and the training image I_(i) are normalized to [0, 1]. ( )_(c∈{r,g,b})represents a pixel color channel, and may include three pixel colorchannels: red, green, and blue (RGB). Because F(I_(i))=S⁻¹×I_(i), I_(i)may be set as the lower limit of S to ensure that the upper limit of allcolor channels of F (I_(i)) after image enhancement is 1, so as toprevent the color from exceeding the color gamut. Setting 1 as the upperlimit of S can avoid erroneously darkening an underexposed region.

In an embodiment, the constraint range of the illumination map in thereconstruction loss function may further be adjusted to meet the actualneeds of different situations. For example, different constraints may beadded to S to adjust the lighting and the color brightness of the image.

By using the reconstruction loss function, the enhanced image obtainedcan be clearer and the contrast of the image can be better. However, ifthe target loss function only includes the reconstruction loss function,there is still the risk of failing to correctly generate contrastdetails and accurate colors of an image.

Therefore, in an embodiment, a local smoothing loss function may furtherbe added to the target loss function to improve the accuracy of imageenhancement.

Specifically, the image enhancement method may further include:

obtaining, based on the image enhancement network model and a trainingimage, a predicted enhanced image corresponding to the training image;

obtaining smoothness loss information between the predicted enhancedimage and a sample enhanced image based on a local smoothing lossfunction, the sample enhanced image being an enhanced imagecorresponding to the training image; and converging the predictedenhanced image and the sample enhanced image based on the smoothnessloss information to obtain a trained image enhancement network model.

In the traditional method, an image is usually enhanced by adjusting ahistogram distribution curve of the image and performing localsmoothness on an illumination map of an optimized image. However, insuch a method, image enhancement is usually performed using asingle-channel illumination map, which leads to deviations in thecontrol of the image color and deficiency in image color enhancement.

Therefore, the three channels, namely, RGB, of the image may beoptimized at the same time, and the illumination map may be learnedusing the learning ability of the network model, to improve the accuracyof image enhancement.

A local smoothing loss function can obtain the smoothness lossinformation of the image, and the local smoothing loss function may beobtained by summing the three channels of an image pixel.

In practical applications, the local smoothing loss function may beobtained by summing the three channels of an image pixel. For example, pmay be used to represent an image pixel, S may be used to represent anillumination map, and a formula for calculating the local smoothing lossfunction L^(i) _(s) may be as follows:

$L_{s}^{i} = {{\sum\limits_{p}{\sum\limits_{c}{\omega_{x,c}^{p}\left( {\partial_{x}S_{p}} \right)}_{c}^{2}}} + {\omega_{y,c}^{p}\left( {\partial_{y}S_{p}} \right)}_{c}^{2}}$

The three channels of the pixel may be summed to obtain the localsmoothing loss function L^(i) _(s), ∂_(x) and ∂_(y) may be used torepresent partial derivatives of horizontal and vertical directions ofthe image space, and ω_(x,c) ^(p) and ω_(y,c) ^(p) may be used torepresent smoothness weights of a spatial change of the three channelsof the pixel. A formula for calculating and ω_(x,c) ^(p) and ω_(y,c)^(p) may be as follows:

ω_(x,c) ^(p)=(|∂_(x) L _(i) ^(p)|_(c) ^(θ)+ε)⁻¹ω_(y,c) ^(p)=(|∂_(y) L_(i) ^(p)|_(c) ^(θ)+ε)⁻¹

L_(i) is a logarithmic image of a training image I_(i), θ=1.2 is aparameter that controls image sensitivity, and ε is a constant, usuallyset to 0.0001 to prevent division by zero.

Training the network model using the local smoothing loss function canreduce overfitting, improve the generalization ability of the networkmodel, and restore good image contrast and clearer details in the image.

In an embodiment, although the Euclidean distance of the chromaticaberration has been implicitly measured in the reconstruction lossfunction, the Euclidean distance measurement can only measure thechromatic aberration numerically, but cannot guarantee that the colorvectors are consistent in direction, which may result in a noticeablecolor mismatch. To accurately restore the color information in theimage, a color loss function may further be introduced.

Specifically, the image enhancement method may further include:

obtaining, based on the image enhancement network model and a trainingimage, a predicted enhanced image corresponding to the training image;

obtaining color loss information between the predicted enhanced imageand a sample enhanced image based on a color loss function, the sampleenhanced image being an enhanced image corresponding to the trainingimage; and converging the predicted enhanced image and the sampleenhanced image based on the color loss information to obtain a trainedimage enhancement network model.

The color loss function can obtain color loss information of an image.For example, the color loss function may be obtained by calculating anincluded angle formed by vectors of the three channels of a pixel of theimage.

In practical applications, the color loss function may be obtainedaccording to an included angle formed by vectors of the three channelsof a pixel of an image. For example, the color loss function can makecolors between the sample enhanced image and the predicted enhancedimage obtained by the network model correspond to each other. For thepredicted enhanced image and the sample enhanced image, the RGB value ofthe image may be regarded as a spatial vector, so as to calculate theincluded angle between the corresponding color channel vectors of thepredicted enhanced image and the sample enhanced image. A smallerincluded angle indicates that directions of the vectors are closer.

In an embodiment, for example, F(I_(i)) may be used to represent thepredicted enhanced image,

may be used to represent the sample enhanced image, and a formula forcalculating the color loss function may be as follows:

$L_{c}^{i} = {\sum\limits_{p}{\angle\left( {\left( {F\left( I_{i} \right)} \right)_{p},{()}_{p}} \right)}}$

In an embodiment, the target loss function may include a reconstructionloss function, a local smoothing loss function, and a color lossfunction. For example, L^(i) _(r) may be used to represent thereconstruction loss function, L^(i) _(s) may be used to represent thelocal smoothing loss function, L^(i) _(c) may be used to represent thecolor loss function, and L may be used to represent the target lossfunction. ω_(r) may be used to represent a weight of the reconstructionloss function in training, ω_(s) may be used to represent a weight ofthe local smoothing loss function in training, and ω_(c) may be used torepresent a weight of the color loss function in training. A formula forcalculating the target loss function may be as follows:

$L = {{\sum\limits_{i = 1}^{N}{\omega_{r}L_{r}^{i}}} + {\omega_{s}L_{s}^{i}} + {\omega_{c}L_{c}^{i}}}$

In an embodiment, for example, during the image enhancement networkmodel training process, ω_(r)=1, ω_(s)=2, and ω_(c)=3.

303: Converge the predicted enhanced image and the sample enhanced imagebased on the loss information to obtain a trained image enhancementnetwork model.

In practical applications, the predicted enhanced image and the sampleenhanced image may be converged based on the loss information to obtaina trained image enhancement network model.

In an embodiment, for example, a loss function may be used to convergethe predicted enhanced image and the sample enhanced image, andcontinuous training may be performed by reducing the error between thepredicted enhanced image and the sample enhanced image, to adjust theweight to an appropriate value. Then the trained image enhancementnetwork model can be obtained.

Training the network model through the image enhancement method andusing the trained network model to enhance the image can speed up theoperation of the network, improve the efficiency of image enhancement,and improve the accuracy of image enhancement without compromising theeffect of enhancement.

The network model trained by the method can realize the customization ofthe image enhancement effect by constraining the illumination. Forexample, the contrast can be enhanced by enhancing the local smoothillumination, setting a preferred exposure level by limiting anillumination degree, and the like.

In an embodiment, the image enhancement method can also adjust theconstraints on the illumination map in the loss function, so that theuser can adjust the image according to a personal preference, such asthe brightness of the image, and the vividness of colors in the image.

In an embodiment, the image enhancement method may also add imagedenoising processing and supplementary generation processing forcompletely lost details in the image to obtain a better enhanced image.

The image enhancement method can be widely used in various imagecapturing conditions, and the image enhancement method can be used toenhance an image taken during the daytime with insufficient dark lightand backlight, or an original image taken at night. The imageenhancement method can also resolve the problem of uneven lightingduring image capturing. As shown in FIG. 8, the original image may beinputted, and the enhanced target image may be directly obtained usingthe image enhancement method. For a 1080P high-definition large image,image enhancement processing can also be performed in real time.Therefore, the image enhancement method can further be extended to imageenhancement for an image in a video.

The image enhancement method can generate a high-quality image. Theenhanced image specifically has clear details, sharp contrast, andmoderate exposure. Problems such as local overexposure or over-darknessare avoided and the color of the image is more vivid and beautiful. Thisimage enhancement method can process images of different pixels. Forexample, a 1080P image can be enhanced in real time, and a 4k-resolutionimage taken by a single-lens reflex (SLR) camera can also be processed.

In an embodiment, the accuracy of the image enhancement method of thisdisclosure is compared with five latest image enhancement methods. Thefive latest image enhancement methods include the Retinex-based imageenhancement method JieP, the deep learning-based image enhancementmethod HDRNet, the deep prior ensemble (DPE), the White-Box, and theDistort-and-Recover. For the foregoing methods, recommended parametersare used for public experiments, and image enhancement results areobtained respectively. The four image enhancement methods based on deeplearning are retrained on a special condition dataset and a standardcondition dataset. Experimental results show that a correct rate of theimage enhancement method of this disclosure is approximately three timesthat of other methods.

In an embodiment, a visual comparison between the image enhancementmethod of this disclosure and other image enhancement methods is furtherperformed. Two special images are used for the visual comparison. One isan unevenly exposed image, which includes imperceptible windmill details(the image comes from a special condition dataset), and the other one isan overall low-light image, which includes a small amount of portraitdetails (the image comes from the standard condition dataset). After thetwo images are enhanced by different image enhancement methods, thevisual comparison is performed. The comparison result shows that theimage enhancement method of this disclosure can restore more details inthe foreground and background, and obtain better contrast withoutsignificantly compromising overexposed or underexposed parts of theimage. Secondly, the image enhancement method of this disclosure candisplay more vivid and natural colors, so that the image effect afterimage enhancement looks more realistic.

In an embodiment, to evaluate the learning efficiency and generalizationability of the deep learning network model, the peak signal to noiseratio (PSNR) and the structural similarity index (SSIM) can be used tomeasure the image enhancement methods. To ensure the accuracy of ameasurement result, network models of all image enhancement methods areretrained on the special condition dataset and the standard conditiondataset. Table 1 shows a comparison of the PSNR and SSIM of the imageenhancement methods after retraining on the special condition datasetand the standard condition dataset. Table 2 shows a comparison of thePSNR and SSIM of the image enhancement methods after retraining on theMIT-Adobe Five K Dataset. As shown in Table 1 and Table 2, the imageenhancement method of this disclosure is superior to other imageenhancement methods, indicating that the image enhancement method ofthis disclosure is not only applicable to the special condition dataset,but also can be extended to the MIT-Adobe Five K Dataset.

TABLE 1 Image enhancement method PSNR SSIM HDRNet 26.33 0.743 DPE 23.580.737 White-Box 21.69 0.718 Distort-and-Recover 24.54 0.712 Imageenhancement method of this 27.02 0.762 disclosure (excludingreconstruction loss function, local smoothing loss function, and colorloss function) Image enhancement method of 28.97 0.783 this disclosure(only including reconstruction loss function) Image enhancement methodof this 30.03 0.822 disclosure (only including reconstruction lossfunction and local smoothing loss function) Image enhancement method ofthis 30.97 0.856 disclosure (including reconstruction loss function,local smoothing loss function, and color loss function)

TABLE 2 Image enhancement method PSNR SSIM HDRNet 28.61 0.866 DPE 24.660.850 White-Box 23.69 0.701 Distort-and-Recover 28.41 0.841 Imageenhancement method of this 28.81 0.867 disclosure (excludingreconstruction loss function, local smoothing loss function, and colorloss function) Image enhancement method of this 29.41 0.871 disclosure(only including reconstruction loss function) Image enhancement methodof this 30.71 0.884 disclosure (only including reconstruction lossfunction and local smoothing loss function) Image enhancement method ofthis 30.80 0.893 disclosure (including reconstruction loss function,local smoothing loss function, and color loss function)

As shown in Table 1 and Table 2, by comparing the image enhancementmethod of this disclosure that includes three loss functions and theimage enhancement method of this disclosure that does not include thethree loss functions in the two tables, it is found that the imageenhancement method of this disclosure learns a mapping from an image toan illumination map better than an image-to-image mapping. In addition,the tables also show different types of loss functions and theimprovement of results, thus proving the role of each loss function.

In an embodiment, user evaluations are further studied to compare theimage enhancement methods. First, 100 images are searched out from animage sharing database through keywords such as “City”, “Flower”,“Food”, “Landscape” and “Portrait”. Pixel intensity of over 50% of theimages is below 0.3. Then a plurality of image enhancement methods areused to perform image enhancement on the images, and participants rateenhancement results corresponding to the image enhancement methods. Toensure the accuracy of the results, the enhancement results arepresented to the participants randomly.

As shown in FIG. 9 to FIG. 14, the participants give scores to sixquestions shown in the drawings, from 1 to 5 points. The six questionsare respectively “Is it easy to recognize details in the image?” “Is theimage bright-colored?” “Is the resulting image visually real?” “Is theresulting image not overexposed?” “Is the resulting image moreattractive than the input image?”, and “What is your total score?” Eachpicture shows the rating of a specific question. The comparison resultsshow that the image enhancement method of this disclosure has achieved ahigher score and is favored by users.

It can be known from the above that, in the embodiments of thisdisclosure, an original image is obtained; synthesis processing isperformed on features of the original image to obtain a firstillumination map corresponding to the original image, a resolution ofthe first illumination map being lower than a resolution of the originalimage; a mapping relationship for mapping an image to a secondillumination map is obtained based on the first illumination map;mapping processing is performed on the original image based on themapping relationship to obtain a second illumination map, a resolutionof the second illumination map being equal to the resolution of theoriginal image; and image enhancement processing is performed on theoriginal image according to the second illumination map to obtain atarget image. This solution enhances an image by deep learning, whichimproves the efficiency and accuracy of image enhancement. Regressionlearning is also performed on the original image and the annotatedillumination map to obtain the network model required for imageenhancement, which makes the training of the network model easier,strengthens the robustness of the network model, and makes it convenientfor further operations on the image. In addition, the three lossfunctions are designed to improve the accuracy of the enhanced image interms of color and contrast. By constraining the illumination map in thenetwork model training process, the image is not over-exposed orover-enhanced.

According to the method described in the foregoing embodiments, thefollowing further provides detailed descriptions using an example.

In this embodiment, as shown in FIG. 4, descriptions are provided usingan example in which the image enhancement apparatus is specificallyintegrated into a network device.

401: A network device obtains an original image.

In practical applications, the network device may obtain original imagesof various image capturing situations for image enhancement. Forexample, the original image may be a normally exposed image, anunderexposed image, an under-lighted image, or a backlit image duringimage capturing. The image enhancement method is not limited to thenormally exposed image, thereby expanding the application scope of theimage enhancement method.

In practical applications, there are many ways for the network device toobtain the original image. For example, the original image may beobtained from local storage, a network side device, or the like.

In an embodiment, for example, when images are captured by a cameradevice, the network device may select a currently captured image as theoriginal image. In another example, when images are captured by a cameradevice and displayed in an image capture interface (such as an imagepreview interface), an image currently displayed on the interface can becropped as the original image.

In an embodiment, the network device may alternatively obtain theoriginal image from a local or external storage unit. For example, anoriginal image may alternatively be obtained from a local imagedatabase.

402: The network device performs synthesis processing on features of theoriginal image to obtain a low-resolution illumination map correspondingto the original image.

Currently, when the deep learning method is used to enhance an image, anetwork model obtained by regression learning from an original image toan annotated image is usually used to perform an image enhancementoperation. However, this method leads to low learning efficiency andpoor robustness of the network model, and defects in image contrast.

In practical applications, the network device may perform the imageenhancement operation using an image enhancement network model obtainedby regression learning from an original image to an illumination map.The image enhancement network model obtained by regression learning fromthe original image to the illumination map has high learning efficiencyand strong robustness, and facilitates further operations on the image.

The image enhancement method is applicable to the image enhancementnetwork model. The image enhancement network model uses a mappingrelationship between the original image and the illumination map toreplace a mapping relationship between the original image and theannotated image. The advantage of this approach is that the mappingbetween the original image and the illumination map usually has arelatively simple form and is known a priori. The image enhancementnetwork model thus has a strong generalization ability, and caneffectively process original images obtained in different situationsunder complex photography conditions.

In practical applications, the low-resolution illumination mapcorresponding to the original image may be obtained through featuresynthesis. The network device may first extract features of the originalimage, and perform feature synthesis on the extracted features togenerate the low-resolution illumination map.

In the traditional method, the method of adjusting a distribution curveof an image histogram is usually used to enhance the image globally.However, this method causes problems such as locally over-brightness,over-exposure, and over-darkness, and the color of the generated imagemay not be very bright.

In addition, to enhance an underexposed image, it is necessary to adjustlocal features (such as contrast, detail definition, shadow andhighlight.) and global features (such as color distribution, averagebrightness, scenario category.) of the image at the same time.Therefore, the accuracy of image enhancement can be improved byseparately extracting the local features and global features of theoriginal image.

In practical applications, the network device may input an originalimage into a convolutional network to extract a local feature and aglobal feature of the original image, and then perform feature synthesison the extracted local feature and global feature to obtain thelow-resolution illumination map.

To improve the accuracy of image feature extraction, a network model maybe used to extract image features.

As shown in FIG. 6, the convolutional network may include a primaryfeature extraction network, a local feature extraction network, and aglobal feature extraction network, the local feature extraction networkbeing connected in parallel with the global feature extraction network,and being connected in series with the primary feature extractionnetwork.

In practical applications, the network device may input the originalimage into the primary feature extraction network including thepre-trained VGG16 network structure to extract the primary feature ofthe original image, then the primary feature is inputted at the sametime into the local feature extraction network and the global featureextraction network that are connected in parallel, and the local featureand the global feature are extracted. The local feature extractionnetwork includes two convolutional layers, and the global featureextraction network includes two convolutional layers and three fullyconnected layers.

To process a high-resolution image in real time, most networkcalculations may be performed in a low-resolution condition, and aresolution of an image may be converted by downsampling.

In practical applications, the network device may obtain a matrix with apreset size of s*s from a matrix composed of pixels of the originalimage, and then convert the pixels in the matrix with the preset sizeinto one pixel. The pixel may be obtained according to a preset rule.For example, the pixel may be an average value of all pixels in thematrix with the preset size. After the pixels of the entire originalimage are converted, a downsampled low-resolution input image can beobtained. After that, the low-resolution input image may be inputtedinto the convolutional network for feature extraction, and subsequentsteps may be performed.

403: The network device obtains, based on the low-resolutionillumination map, a mapping transformation matrix for mapping an imageto a second illumination map.

For example, the network device may perform spatial-domain andrange-domain sampling on the pixels of the low-resolution illuminationmap to obtain the sampled pixels, then find positions of thecorresponding pixels in the grid, and perform a grid differenceoperation to obtain the mapping transformation matrix.

404: The network device performs mapping processing on the originalimage based on the mapping transformation matrix to obtain anoriginal-resolution illumination map.

To improve the accuracy of image enhancement, mapping processing may beperformed on the original image in a bilateral grid manner.

In practical applications, as shown in FIG. 7, the network device mayobtain a mapping transformation matrix according to the low-resolutionillumination map 705, and then perform mapping processing on theoriginal image 701 using the mapping relationship to obtain a mappedimage, which is a low-resolution image. After that, bilateral gridupsampling 706 may be performed on the mapped image, and based on pixelsof the mapped image, a suitable interpolation algorithm is used tointerpolate new elements between the pixels to obtain theoriginal-resolution illumination map 707.

405: The network device performs image enhancement processing on theoriginal image according to the original-resolution illumination map toobtain a target image.

The problem of image enhancement can be regarded as a problem of findinga mapping relationship between the original image and the target image.

may be used to represent a matrix corresponding to the target image, Imay be used to represent a matrix corresponding to the original image,and a function F may be used to represent a mapping function between theoriginal image and the target image. Then the mapping function F may beexpressed by the following formula:

=F(I)

The target image, the original image, and the original-resolutionillumination map are related to each other. S may be used to represent amatrix corresponding to the original-resolution illumination map,

may be used to represent the matrix corresponding to the target image,and I may be used to represent the matrix corresponding to the originalimage. Then the relationship among the target image, the original image,and the original-resolution illumination map is as follows:

I=S*

Therefore, the network device can obtain the target image according tothe original image and the original-resolution illumination map.According to the original image I and the original-resolutionillumination map S, the obtained target image

is shown in the following formula:

F(I)=S ⁻¹ ·I

In practical applications, as shown in FIG. 7, the network device firstobtains the original image, and downsamples the original image to obtaina low-resolution input image of 256×256 pixels. Then the low-resolutioninput image is inputted into the primary feature extraction network 702including the pre-trained VGG16 network model to extract the primaryfeature of the original image, then the primary feature is inputtedseparately into the local feature extraction network 703 and the globalfeature extraction network 704 that are connected in parallel, and thelocal feature and the global feature are extracted and merged to obtainthe low-resolution illumination map 705. Then, the mappingtransformation matrix is obtained through bilateral grid upsampling 706,and the original-resolution illumination map 707 is obtained accordingto the mapping transformation matrix. Finally, the target image 709 isobtained through the formula I=S*

. The process of image enhancement 708 can be accelerated by this imageenhancement method, thereby improving the efficiency of imageenhancement.

In practical applications, the image enhancement method further includesa training process of an image enhancement network model. The imageenhancement method further includes the following procedure.

A: The network device obtains, based on an image enhancement networkmodel and a training image, a predicted enhanced image corresponding tothe training image.

In practical applications, the network device may input the trainingimage into the image enhancement network model to obtain the predictedenhanced image corresponding to the training image. The imageenhancement method for enhancing a training image through the imageenhancement network model is equal to the image enhancement method forenhancing an original image through the image enhancement network model,which has been described above and is not repeated here.

In practical applications, the network device may increase the diversityof training samples by randomly cropping the training image. Thetraining image may be randomly cropped into a plurality of images of512×512 pixels to increase the diversity of samples.

The accuracy of the network model can be improved by increasing thediversity of the training image. The training image may include imagesin various image capturing situations, such as an image of normalexposure, underexposure, insufficient light, or backlight. The networkmodel trained based on such training images can adapt to images obtainedin different capturing situations in reality.

In practical applications, by obtaining a standard condition dataset anda special condition dataset, the network device may construct trainingimages including a plurality of image capturing types according to thestandard condition dataset and the special condition dataset. Thestandard condition dataset is a dataset that includes normally exposedimages. The standard condition dataset be the MIT-Adobe Five K Dataset,and annotations of Expert C are selected as annotations of the trainingsamples in the dataset. However, because the standard condition datasetis created mainly to enhance general images rather than underexposedimages, the standard condition dataset only includes a small portion(about 4%) of unexposed images. As a result, the standard conditiondataset lacks images taken in special image capturing conditions, suchas images captured at night or images obtained under non-uniformlighting conditions. To increase the diversity of samples, a specialcondition dataset is introduced.

The special condition dataset is a dataset that includes abnormallyexposed images. For example, the special condition dataset may includeimages captured in special image capturing conditions, such asunderexposure, insufficient light, or backlight. Such a specialcondition dataset may include various image capturing situations,scenes, themes, and styles. The added special condition dataset cansupplement image types that the standard condition dataset lacks.

Training the network model based on the training images constructed bythe standard condition dataset and the special condition dataset canmake the trained network model adapt to various image capturingsituations, thereby improving the accuracy of image enhancement.

B: The network device obtains loss information between the predictedenhanced image and a sample enhanced image based on a target lossfunction.

In practical applications, the network device may obtain the lossinformation between the predicted enhanced image and the sample enhancedimage through the target loss function. The loss information may be adifference between the predicted enhanced image and the sample enhancedimage, and may be reduced by training the network model.

At present, an image is usually enhanced by adjusting an illuminationmap of the image and performing local smoothing optimization operationson the illumination map. However, such a method may leave traces ofmanual changes of the halo and cause local overexposure of the image,resulting in excessive image enhancement. Therefore, a target lossfunction including a reconstruction loss function, a local smoothingloss function, and a color loss function may be designed. Byconstraining the illumination map, the image is not to be over-exposedor over-enhanced.

In practical applications, the target loss function includes areconstruction loss function, a local smoothing loss function, and acolor loss function. L^(i) _(r) is used to represent the reconstructionloss function, L^(i) _(s) is used to represent the local smoothing lossfunction, L^(i) _(c) is used to represent the color loss function, and Lis used to represent the target loss function. ω_(r) is used torepresent a weight of the reconstruction loss function in training,ω_(s) is used to represent a weight of the local smoothing loss functionin training, and ω_(c) is used to represent a weight of the color lossfunction in training. A formula for calculating the target loss functionis as follows:

$L = {{\sum\limits_{i = 1}^{N}{\omega_{r}L_{r}^{i}}} + {\omega_{s}L_{s}^{i}} + {\omega_{c}L_{c}^{i}}}$

In practical applications, during the image enhancement network modeltraining process, ω_(r)=1, ω_(s)=2, and ω_(c)=3.

In practical applications, the network device may use S to represent anoriginal-resolution illumination map matrix corresponding to thepredicted enhanced image,

to represent the sample enhanced image, and I_(i) to represent thetraining image. The network device may multiply the original-resolutionillumination map matrix S corresponding to the predicted enhanced imageby the sample enhanced image

to calculate a Euclidean distance error metric with the training imageI_(i) to obtain the reconstruction loss function. A formula of thereconstruction loss function L^(i) _(r) may be as follows:

L ^(i) _(r) =|I _(i) −S×

∥ ²

A multi-channel illumination range may be (I_(i))_(c)≤(S)_(c)≤1, allpixel channels in the sample enhanced image

and the training image I_(i) are normalized to [0, 1]. ( )_(cε{r,g,b})represents a pixel color channel, and may include three pixel colorchannels: red, green, and blue (RGB). Because F(I_(i))=S⁻¹×I_(i), I_(i)may be set as the lower limit of S to ensure that the upper limit of allcolor channels of F(I_(i)) after image enhancement is 1, so as toprevent the color from exceeding the color gamut. Setting 1 as the upperlimit of S can avoid erroneously darkening an underexposed region.

In practical applications, the network device may further adjust theconstraint range of the illumination map in the reconstruction lossfunction to meet the actual needs of different situations. The networkdevice may add different constraints to S to adjust the brightness,vividness of the colors of the image, and the like.

Using the reconstruction loss function, the enhanced image obtained canbe clearer and the contrast of the image can be better. However, if thetarget loss function only includes the reconstruction loss function,there is still the risk of failing to correctly generate contrastdetails and accurate colors of an image.

In the traditional method, an image is usually enhanced by adjusting ahistogram distribution curve of the image and performing localsmoothness on an illumination map of an optimized image. However, insuch a method, image enhancement is usually performed using asingle-channel illumination map, which leads to deviations in thecontrol of the image color and deficiency in image color enhancement.

Therefore, the three channels, namely, RGB, of the image may beoptimized at the same time, and the illumination map may be learnedusing the learning ability of the network model, to improve the accuracyof image enhancement.

A local smoothing loss function can obtain the smoothness lossinformation of the image, and the local smoothing loss function can beobtained by summing the three channels of an image pixel.

In practical applications, the network device obtains the localsmoothing loss function by summing the three channels of an image pixel.Using p to represent an image pixel and S to represent an illuminationmap, a formula for calculating the local smoothing loss function 1″, maybe as follows:

$L_{s}^{i} = {{\sum\limits_{p}{\sum\limits_{c}{\omega_{x,c}^{p}\left( {\partial_{x}S_{p}} \right)}_{c}^{2}}} + {\omega_{y,c}^{p}\left( {\partial_{y}S_{p}} \right)}_{c}^{2}}$

The network device may sum the three channels of the pixel to obtain thelocal smoothing loss function L^(i) _(s), use ∂_(x) and ∂_(y) torepresent partial derivatives of horizontal and vertical directions ofthe image space, and use ωhd x,c^(p) and ωhd y,c^(p) to representsmoothness weights of a spatial change of the three channels of thepixel. A formula for calculating ωhd x,c^(p) and ωhd y,c^(p) may be asfollows:

ω_(x,c) ^(p)=(|∂_(x) L _(i) ^(p)|_(c) ^(θ)+ε)⁻¹ω_(y,c) ^(p)=(|∂_(y) L_(i) ^(p)|_(c) ^(θ)+ε)⁻¹

L_(i) is a logarithmic image of a training image I_(i), θ=1.2 is aparameter that controls image sensitivity, and ε is a constant, usuallyset to 0.0001 to prevent division by zero.

Training the network model using the local smoothing loss function canreduce overfitting, improve the generalization ability of the networkmodel, and restore good image contrast and clearer details in the image.

Although the Euclidean distance of the chromatic aberration has beenimplicitly measured in the reconstruction loss function, the Euclideandistance measurement can only measure the chromatic aberrationnumerically, but cannot guarantee that the color vectors are consistentin direction, which may result in a noticeable color mismatch. Toaccurately restore the color information in the image, a color lossfunction may further be introduced.

The color loss function can obtain color loss information of an image.The color loss function may be obtained by calculating an included angleformed by vectors of the three channels of a pixel of the image.

In practical applications, the network device may obtain the color lossfunction according to an included angle formed by vectors of the threechannels of a pixel of an image. For example, the color loss functioncan make colors between the sample enhanced image and the predictedenhanced image obtained by the network model correspond to each other.For the predicted enhanced image and the sample enhanced image, the RGBvalue of the image may be regarded as a spatial vector, so as tocalculate the included angle between the corresponding color channelvectors of the predicted enhanced image and the sample enhanced image.The smaller the included angle, the closer the directions between thevectors.

In practical applications, F(I_(i)) is used to represent the predictedenhanced image,

is used to represent the sample enhanced image, and a formula forcalculating the color loss function L^(i) _(c) may be as follows:

$L_{c}^{i} = {\sum\limits_{p}{\angle\left( {\left( {F\left( I_{i} \right)} \right)_{p},{()}_{p}} \right)}}$

C: The network device converges the predicted enhanced image and thesample enhanced image based on the loss information to obtain a trainedimage enhancement network model.

In practical applications, the network device may converge the predictedenhanced image and the sample enhanced image based on the lossinformation to obtain a trained image enhancement network model.

In practical applications, the network device may converge the predictedenhanced image and the sample enhanced image using the reconstructionloss function, the local smoothing loss function, and the color lossfunction. Continuous training may be performed by reducing the errorbetween the predicted enhanced image and the sample enhanced image, toadjust the weight to an appropriate value. Then the trained imageenhancement network model can be obtained.

Training the network model through the image enhancement method andusing the trained network model to enhance the image can speed up theoperation of the network, improve the efficiency of image enhancement,and improve the accuracy of image enhancement without compromising theeffect of enhancement.

The network model trained by the method can realize the customization ofthe image enhancement effect by constraining the illumination. Forexample, the contrast can be enhanced by enhancing the local smoothillumination, setting a preferred exposure level by limiting anillumination degree, and the like.

In an embodiment, the image enhancement method can also adjust theconstraints on the illumination map in the loss function, so that theuser can adjust the image according to a personal preference, such asthe brightness of the image, and the vividness of colors in the image.

In an embodiment, the image enhancement method may also add imagedenoising processing and supplementary generation processing forcompletely lost details in the image to obtain a better enhanced image.

In practical applications, the image enhancement method needs to beprovided with a graphics processing unit (GPU) that meets a performancerequirement and needs to be configured with the TensorFlow deep learningplatform, on which the image enhancement method can be directlyoperated.

The image enhancement method can be widely used in various imagecapturing conditions, and the image enhancement method can be used toenhance an image taken during the daytime with insufficient dark lightand backlight, or an original image taken at night. The imageenhancement method can also resolve the problem of uneven lightingduring image capturing. As shown in FIG. 8, the original image may beinputted, and the enhanced target image may be directly obtained usingthe image enhancement method. For a 1080P high-definition large image,image enhancement processing can also be performed in real time.Therefore, the image enhancement method can further be extended to imageenhancement for an image in a video.

The image enhancement method can generate a high-quality image. Theenhanced image specifically has clear details, sharp contrast, andmoderate exposure. Problems such as local overexposure or over-darknessare avoided and the color of the image is more vivid and beautiful. Thisimage enhancement method can process images of different pixels. Forexample, a 1080P image can be enhanced in real time, and a 4k-resolutionimage taken by an SLR camera can also be processed.

It can be learned from the above, in the embodiments of this disclosure,the network device can obtain an original image; perform synthesisprocessing on features of the original image to obtain a low-resolutionillumination map corresponding to the original image; obtain, based onthe low-resolution illumination map, a mapping transformation matrix formapping an image to a second illumination map; perform mappingprocessing on the original image based on the mapping transformationmatrix to obtain an original-resolution illumination map; and performimage enhancement processing on the original image according to theoriginal-resolution illumination map to obtain a target image. Thissolution enhances an image by deep learning, which improves theefficiency and accuracy of image enhancement. Regression learning isalso performed on the original image and the annotated illumination mapto obtain the network model required for image enhancement, which makesthe training of the network model easier, strengthens the robustness ofthe network model, and makes it convenient for further operations on theimage. In addition, the three loss functions are designed to improve theaccuracy of the enhanced image in terms of color and contrast. Byconstraining the illumination map in the network model training process,the image is not to be over-exposed or over-enhanced.

To better implement the foregoing method, an embodiment of thisdisclosure further provides an image enhancement apparatus, which may beintegrated in a network device.

For example, as shown in FIG. 15, the image enhancement apparatus mayinclude an obtaining module 151, a feature synthesis module 152, amapping relationship obtaining module 153, a mapping module 154, and animage enhancement module 155.

The obtaining module 151 is configured to obtain an original image.

The feature synthesis module 152 is configured to perform synthesisprocessing on features of the original image to obtain a firstillumination map corresponding to the original image, a resolution ofthe first illumination map being lower than a resolution of the originalimage.

The mapping relationship obtaining module 153 is configured to obtain,based on the first illumination map, a mapping relationship for mappingan image to a second illumination map.

The mapping module 154 is configured to perform mapping processing onthe original image based on the mapping relationship to obtain a secondillumination map, a resolution of the second illumination map beingequal to the resolution of the original image.

The image enhancement module 155 is configured to perform imageenhancement processing on the original image according to the secondillumination map to obtain a target image.

In an embodiment, referring to FIG. 16, the feature synthesis module 152may include:

a feature extraction submodule 1521, configured to extract a localfeature and a global feature of the original image based on aconvolutional network; and

a feature synthesis submodule 1522, configured to perform featuresynthesis on the local feature and the global feature to obtain thefirst illumination map corresponding to the original image.

During specific implementation, the foregoing units may be implementedas independent entities, may be combined in different manners, or may beimplemented as the same entity or several entities. For specificimplementation of the foregoing units, refer to the foregoing methodembodiments. Details are not described herein again.

It can be known from the above that, in the embodiments of thisdisclosure, the obtaining module 151 obtains an original image; thefeature synthesis module 152 performs synthesis processing on featuresof the original image to obtain a first illumination map correspondingto the original image, a resolution of the first illumination map beinglower than a resolution of the original image; the mapping relationshipobtaining module 153 obtains, based on the first illumination map, amapping relationship for mapping an image to a second illumination map;the mapping module 154 performs mapping processing on the original imagebased on the mapping relationship to obtain a second illumination map, aresolution of the second illumination map being equal to the resolutionof the original image; and the image enhancement module 155 performsimage enhancement processing on the original image according to thesecond illumination map to obtain a target image. This solution enhancesan image by deep learning, which improves the efficiency and accuracy ofimage enhancement. Regression learning is also performed on the originalimage and the annotated illumination map to obtain the network modelrequired for image enhancement, which makes the training of the networkmodel easier, strengthens the robustness of the network model, and makesit convenient for further operations on the image. In addition, thethree loss functions are designed to improve the accuracy of theenhanced image in terms of color and contrast. By constraining theillumination map in the network model training process, the image is notto be over-exposed or over-enhanced.

An embodiment of this disclosure further provides a computer device,which may be a server, a terminal or another device. The computer deviceis integrated with any image enhancement apparatus provided in theembodiments of this disclosure, such as the network device describedabove. FIG. 17 is a schematic structural diagram of a computer deviceaccording to an embodiment of this disclosure.

Specifically, the computer device may include components such as aprocessor 171 including one or more processing cores, a memory 172including one or more computer-readable storage media, a power supply173, and an input unit 174. A person skilled in the art may understandthat the structure of the computer device shown in FIG. 17 does notconstitute a limitation to the network device, and the device mayinclude more components or fewer components than those shown in thefigure, or some components may be combined, or a different componentdeployment may be used.

The processor 171 is a control center of the computer device, andconnects various parts of the entire computer device using variousinterfaces and lines. By running or executing software programs and/ormodules stored in the memory 172, and invoking data stored in the memory172, the processor performs various functions and data processing of thecomputer device, thereby performing overall monitoring on the computerdevice. Optionally, the processor 171 may include one or more processingcores. Preferably, the processor 171 may integrate an applicationprocessor and a modem processor. The application processor mainlyprocesses an operating system, a user interface, an application program,and the like, and the modem processor mainly processes wirelesscommunication. It may be understood that alternatively, the modemprocessor may not be integrated into the processor 171.

The memory 172 may be configured to store a software program and amodule, and the processor 171 runs the software program and the modulethat are stored in the memory 172, to implement various functionalapplications and data processing. The memory 172 may mainly include aprogram storage area and a data storage area. The program storage areamay store an operating system, an application program required by atleast one function (such as a sound playing function and an imagedisplay function), and the like. The data storage area may store datacreated according to use of the network device, and the like. Inaddition, the memory 172 may include a high-speed random access memory,and may further include a non-volatile memory, such as at least onemagnetic disk storage device, a flash memory device or othernon-volatile solid state storage devices. Correspondingly, the memory172 may further include a memory controller, so that the processor 171can access the memory 172.

The computer device further includes the power supply 173 supplyingpower to the components. The power supply 173 may be logically connectedto the processor 171 using a power management system, therebyimplementing functions such as charging, discharging, and powerconsumption management using the power management system. The powersupply 173 may further include one or more of a direct current oralternating current power supply, a re-charging system, a power failuredetection circuit, a power supply converter or inverter, a power supplystate indicator, and any other component.

The computer device may further include the input unit 174. The inputunit 174 may be configured to receive input digit or characterinformation and generate keyboard, mouse, joystick, optical, ortrackball signal input related to user settings and function control.

Although not shown in the figure, the computer device may furtherinclude a display unit, and the like. Details are not described hereinagain. Specifically, in this embodiment, the processor 171 in thecomputer device may load executable files corresponding to processes ofone or more application programs to the memory 172 according to thefollowing instructions, and the processor 171 runs the applicationprograms stored in the memory 172, to implement various functions:

obtaining an original image; performing synthesis processing on featuresof the original image to obtain a first illumination map correspondingto the original image, a resolution of the first illumination map beinglower than a resolution of the original image; obtaining, based on thefirst illumination map, a mapping relationship for mapping an image to asecond illumination map; performing mapping processing on the originalimage based on the mapping relationship to obtain a second illuminationmap, a resolution of the second illumination map being equal to theresolution of the original image; and performing image enhancementprocessing on the original image according to the second illuminationmap to obtain a target image.

For specific implementations of the above operations, refer to theforegoing embodiments. Details are not described herein again.

It can be known from the above that, in the embodiments of thisdisclosure, an original image is obtained; synthesis processing isperformed on features of the original image to obtain a firstillumination map corresponding to the original image, a resolution ofthe first illumination map being lower than a resolution of the originalimage; a mapping relationship for mapping an image to a secondillumination map is obtained based on the first illumination map;mapping processing is performed on the original image based on themapping relationship to obtain a second illumination map, a resolutionof the second illumination map being equal to the resolution of theoriginal image; and image enhancement processing is performed on theoriginal image according to the second illumination map to obtain atarget image. This solution enhances an image by deep learning, whichimproves the efficiency and accuracy of image enhancement. Regressionlearning is also performed on the original image and the annotatedillumination map to obtain the network model required for imageenhancement, which makes the training of the network model easier,strengthens the robustness of the network model, and makes it convenientfor further operations on the image. In addition, the three lossfunctions are designed to improve the accuracy of the enhanced image interms of color and contrast. By constraining the illumination map in thenetwork model training process, the image is not to be over-exposed orover-enhanced.

A person of ordinary skill in the art may understand that, all or somesteps of the methods in the foregoing embodiments may be implementedusing instructions, or implemented through instructions controllingrelevant hardware, and the instructions may be stored in acomputer-readable storage medium and loaded and executed by a processor.

Accordingly, an embodiment of this disclosure provides a storage medium,storing a plurality of instructions. The instructions can be loaded by aprocessor, to perform the steps in any image enhancement methodaccording to the embodiments of this disclosure. For example, theinstructions may perform the following steps:

obtaining an original image; performing synthesis processing on featuresof the original image to obtain a first illumination map correspondingto the original image, a resolution of the first illumination map beinglower than a resolution of the original image; obtaining, based on thefirst illumination map, a mapping relationship for mapping an image to asecond illumination map; performing mapping processing on the originalimage based on the mapping relationship to obtain a second illuminationmap, a resolution of the second illumination map being equal to theresolution of the original image; and performing image enhancementprocessing on the original image according to the second illuminationmap to obtain a target image.

For specific implementations of the above operations, refer to theforegoing embodiments. Details are not described herein again.

The storage medium may include a read-only memory (ROM), a random accessmemory (RAM), a magnetic disk, an optical disc, or the like.

Because the instructions stored in the storage medium may perform thesteps of any image enhancement method provided in the embodiments ofthis disclosure, the instructions can implement beneficial effects thatcan be implemented by any image enhancement method provided in theembodiments of this disclosure. For details, reference may be made tothe foregoing embodiments. Details are not described herein again.

The image enhancement method and apparatus, and the storage mediumprovided in the embodiments of this disclosure are described above indetail. Although the principles and implementations of this disclosureare described using specific examples in this specification, thedescriptions of the foregoing embodiments are merely intended to helpunderstand the method and the core idea of the method of thisdisclosure. Meanwhile, a person skilled in the art may makemodifications to the specific implementations and application rangeaccording to the idea of this disclosure. In conclusion, the content ofthis specification is not to be construed as a limitation to thisdisclosure.

What is claimed is:
 1. An image enhancement method, performed by anetwork device, comprising: obtaining an original image; performingsynthesis processing on features of the original image to obtain a firstillumination map corresponding to the original image, a resolution ofthe first illumination map being lower than a resolution of the originalimage; obtaining, based on the first illumination map, a mappingrelationship between an image and an illumination map; performingmapping processing on the original image based on the mappingrelationship to obtain a second illumination map, a resolution of thesecond illumination map being equal to the resolution of the originalimage; and performing image enhancement processing on the original imageaccording to the second illumination map to obtain a target image. 2.The image enhancement method of claim 1, wherein the performing thesynthesis processing on the features of the original image to obtain thefirst illumination map comprises: extracting a local feature and aglobal feature of the original image using a convolutional network; andperforming feature synthesis on the local feature and the global featureto obtain the first illumination map corresponding to the originalimage.
 3. The image enhancement method of claim 2, wherein theextracting the local feature and the global feature of the originalimage using the convolutional network comprises: inputting the originalimage to the convolutional network, the convolutional network comprisinga primary feature extraction network, a local feature extractionnetwork, and a global feature extraction network, the local featureextraction network and the global feature extraction network beingconnected in parallel, and the local feature extraction network and theglobal feature extraction network being connected in series with theprimary feature extraction network; performing a convolution operationon the original image using the primary feature extraction network toextract a primary feature of the original image; performing aconvolution operation on the primary feature using the local featureextraction network to extract the local feature; and performing aconvolution operation on the primary feature using the global featureextraction network to extract the global feature.
 4. The imageenhancement method of claim 1, wherein the performing the synthesisprocessing on the features of the original image to obtain the firstillumination map comprises: downsampling pixels of the original image toobtain an input image; and performing synthesis processing on featuresof the input image to obtain the first illumination map corresponding tothe original image.
 5. The image enhancement method of claim 1, whereinthe performing the mapping processing on the original image based on themapping relationship to obtain the second illumination map comprises:performing mapping processing on the original image based on the mappingrelationship to obtain a mapped image; and upsampling the mapped imageto obtain the second illumination map.
 6. The image enhancement methodof claim 1, wherein the obtaining the mapping relationship comprises:obtaining, based on a preset training image and a sample enhanced imagecorresponding to the preset training image, the mapping relationshipthat enables contrast loss information between a predicted enhancedimage corresponding to the preset training image and the sample enhancedimage to meet a preset condition, the predicted enhanced image being anenhanced image obtained by performing mapping processing on the presettraining image based on the mapping relationship.
 7. The imageenhancement method of claim 6, wherein the contrast loss information isobtained by calculating an Euclidean distance between the predictedenhanced image and the sample enhanced image.
 8. The image enhancementmethod of claim 1, wherein the obtaining the mapping relationshipcomprises: obtaining, based on a preset training image and a sampleenhanced image corresponding to the preset training image, the mappingrelationship that enables smoothness loss information between apredicted enhanced image corresponding to the preset training image andthe sample enhanced image to meet a preset condition, the predictedenhanced image being an enhanced image obtained by performing mappingprocessing on the preset training image based on the mappingrelationship.
 9. The image enhancement method of claim 8, wherein thesmoothness loss information is obtained by summing spatial variations ofvalues of three color channels at each pixel of the mappingrelationship.
 10. The image enhancement method of claim 1, wherein theobtaining a mapping relationship for mapping an image to a secondillumination map comprises: obtaining, based on a preset training imageand a sample enhanced image corresponding to the preset training image,the mapping relationship that enables color loss information between apredicted enhanced image corresponding to the preset training image andthe sample enhanced image to meet a preset condition, the predictedenhanced image being an enhanced image obtained by performing mappingprocessing on the preset training image based on the mappingrelationship.
 11. The image enhancement method of claim 10, wherein thecolor loss information is obtained by similarities between color vectorsof each pixel in the predicted enhanced image and in the sample enhancedimage.
 12. An image enhancement apparatus, comprising: a memory operableto store computer-readable instructions; and a processor operable toread the computer-readable instructions, the processor, when executingthe computer-readable instructions, is configured to: obtain an originalimage; perform synthesis processing on features of the original image toobtain a first illumination map corresponding to the original image, aresolution of the first illumination map being lower than a resolutionof the original image; obtain, based on the first illumination map, amapping relationship between an image and an illumination map; performmapping processing on the original image based on the mappingrelationship to obtain a second illumination map, a resolution of thesecond illumination map being equal to the resolution of the originalimage; and perform image enhancement processing on the original imageaccording to the second illumination map to obtain a target image. 13.The image enhancement apparatus of claim 12, wherein the processor isconfigured to: extract a local feature and a global feature of theoriginal image using a convolutional network; and perform featuresynthesis on the local feature and the global feature to obtain thefirst illumination map corresponding to the original image.
 14. Theimage enhancement apparatus of claim 13, wherein the processor isconfigured to: input the original image to the convolutional network,the convolutional network comprising a primary feature extractionnetwork, a local feature extraction network, and a global featureextraction network, the local feature extraction network and the globalfeature extraction network being connected in parallel, and the localfeature extraction network and the global feature extraction networkbeing connected in series with the primary feature extraction network;perform a convolution operation on the original image using the primaryfeature extraction network to extract a primary feature of the originalimage; perform a convolution operation on the primary feature using thelocal feature extraction network to extract the local feature; andperform a convolution operation on the primary feature using the globalfeature extraction network to extract the global feature.
 15. The imageenhancement apparatus of claim 12, wherein the processor is configuredto: downsample pixels of the original image to obtain an input image;and perform synthesis processing on features of the input image toobtain the first illumination map corresponding to the original image.16. The image enhancement apparatus of claim 12, wherein the processoris configured to: perform mapping processing on the original image basedon the mapping relationship to obtain a mapped image; and upsample themapped image to obtain the second illumination map.
 17. The imageenhancement apparatus of claim 12, wherein the processor is configuredto: obtain, based on a preset training image and a sample enhanced imagecorresponding to the preset training image, the mapping relationshipthat enables contrast loss information between a predicted enhancedimage corresponding to the preset training image and the sample enhancedimage to meet a preset condition, the predicted enhanced image being anenhanced image obtained by performing mapping processing on the presettraining image based on the mapping relationship.
 18. The imageenhancement apparatus of claim 12, wherein the processor is configuredto: obtain, based on a preset training image and a sample enhanced imagecorresponding to the preset training image, the mapping relationshipthat enables smoothness loss information between a predicted enhancedimage corresponding to the preset training image and the sample enhancedimage to meet a preset condition, the predicted enhanced image being anenhanced image obtained by performing mapping processing on the presettraining image based on the mapping relationship.
 19. The imageenhancement apparatus of claim 12, wherein the processor is configuredto: obtaining, based on a preset training image and a sample enhancedimage corresponding to the preset training image, the mappingrelationship that enables color loss information between a predictedenhanced image corresponding to the preset training image and the sampleenhanced image to meet a preset condition, the predicted enhanced imagebeing an enhanced image obtained by performing mapping processing on thepreset training image based on the mapping relationship.
 20. Anon-transitory computer-readable storage medium, having processorexecutable instructions stored thereon for causing a processor toperform operations comprising: obtaining an original image; performingsynthesis processing on features of the original image to obtain a firstillumination map corresponding to the original image, a resolution ofthe first illumination map being lower than a resolution of the originalimage; obtaining, based on the first illumination map, a mappingrelationship between an image and an illumination map; performingmapping processing on the original image based on the mappingrelationship to obtain a second illumination map, a resolution of thesecond illumination map being equal to the resolution of the originalimage; and performing image enhancement processing on the original imageaccording to the second illumination map to obtain a target image.